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ABSTRACT 

The Austin Independent School District (Texas) 
developed the Accelerating Literacy Program (ALP) in 1993 to provide 
additional support for students who were having trouble passing the 
Texas Assessment of Academic Skills (TAAS) in the elementary grades. 
The ALP operates on the principle of integrating reading, writing, 
listening, and speaking activities into a whole language approach to 
instruction. It seeks to accelerate the learning process by building 
on student successes. The 13 campuses involved in the ALP were 
selected on the basis of poor performance by their fourth grade TAAS 
takers. Schools and teachers were allowed great latitude in designing 
a course of instruction to meet the needs of students through ALP. 
Overall, the program seems to have met most of its goals. All of the 
28 teachers surveyed indicated that the students made somewhat more 
or much more progress than they would have in a typical period of 
instruction. Teachers were pleased with the training they received 
and with the portfolio approach used in the program. Records 
indicated that ALP students increased their reading text level and 
made s tat is t ical ly significant gains in overall reading skills. The 
majority of teachers (92.87.) indicated that they would continue to 
use program methods and materials. Recommendations for program 
improvement include improved parent training and participation, 
better resource allocation, and better running record keeping. 
Appendixes present tables of teacher and par ent~t rainer survey 
responses. (Contains 7 figures, 21 appendix tables, and 30 
references . ) (SLD) 
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Program Description 



The Austin Independent School District (AISD) 
developed the Accelerating Literacy Program in 
1 993 to provide additional support for students who 
were having trouble passing the Texas Assessment 
of Academic Skills (TAAS) in the elementary 
grades. At the time the program was developed, 13 
Austin schools were either on the AISD C list (i.e., 
less than 25% of students passing all sections of the 
most recent TAAS) or on the Texas Education 
Agency’s (TEA’s) list of schools that are "clearly 
unacceptable" (less than 20% of students passing all 
sections of the TAAS). Often students who do not 
pass the TAAS have started school without the same 
academic foundation to which some of their peers 
have had access. Failure for these schools begins 
much earlier than the fourth grade, when the schools 
do not bridge the gap in earlier grades between what 
the students know and what they need to know. 

The Accelerating Literacy Program (ALP) as 
developed by Evaluation Consultant Services (ECS) 
was chosen for implementation in the 1993-94 
summer session because of its reported success in a 
comparable environment (i.e., the South Bay (CA) 
Union School District). The curriculum is flexible 
enough that it can be used as a stand-alone program 
(such as AISD’s program) or as an adjunct to regular 
school year reading instruction. 

The thematic principle upon which the program 
operates is that of integrating reading, writing, 
listening, and speaking activities into a whole 
language approach to instruction. The program 
seeks to "accelerate” the learning process by building 
on students' successes. According to the theory of 
the program, when a teacher determines that a 
student has mastered text at a given level, the teacher 
should immediately challenge the student at a higher 
leveL According to the ALP instruction manual, this 
objective is pursued through the following steps: 

• Evaluating the reading process through mis cue 
analysis; 

• Applying the four types of reading in whole 
language instruction; 0 * 



The 13 campuses involved in the Accelerating 
Literacy Program were selected on the basis of poor 
performance by their fourth-grade TAAS takers. 

Each campus, and in fact each teacher, was allowed 
great latitude in designing a course of instruction to 
meet the needs of the students. The level of 
coordination between classes varied from campus to 
campus, although in most cases periodic staff 
meetings were held. During these meetings, the lead 
teacher could facilitate strategy sessions, answer 
questions, and distribute materials. The frequency 
of these meetings was determined by the staff. 

Major Findings 

1 . Overall, the program seems to have met most of 
its goals. All of the teachers surveyed indicated 
that the students made "somewhat more" (45%) 
or "much more" (55%) progress than they 
would have in a typical four-week period 
during the regular school year. Teachers were 
generally pleased with the training that they 
received, and the portfolios were used 
successfully in the program. Teachers reported 
96.3% of the time that portfolios were accurate 
representations of the students' achievement 
during the program, (p. 18) 

2. In spite of some problems with data collection, 
the running records indicate that ALP students 
increased their reading text level an average of 
2.93 reading levels, (p.14) 

3. Analysis of the ALP Developmental Checklist 
of Learning Indicators indicated that ALP 
students made statistically si gnificant gains in 
their overall reading skills for the second 
consecutive year. (p. 14) 



7. Campus -based program managers were 
not the norm during the regular school 
year, as a result, some of the prepro- 
gram planning was not adequately 
handled, (p. 19) 

Recommendations 

1 . In the future, it would help to have someone 
with the authority and responsibilities for 
planning the programs at each campus year 
round. Also, at least a half-time coordinator 
should be added to the program budget to 
work more closely with the individual 
campus programs during the year. 

2. More attention should be paid to how 
resources best can be provided for all 
campuses. Part of a planning meeting could 
be dedicated to cross-campus discussion on 
this subject. Such a meeting should be held 
early in the planning process and commit- 
ments should be obtained from principals to 
make a certain level of resources available to 
the program staff during the summer. 

3. The program could benefit from evolving to 
four longer classroom days with the fifth day 
being used for planning, and making parent 
contact. 

4. It would also help the teacher-support aspect 
of the program if there were planning days 
after the program ended which teachers 
could use to debrief and develop strategy on 
transferring the program to their regular 
classrooms. 

5. The problems with the use of running 
records indicate that many teachers need 
more gui d an ceA raining to use this 
assessment instrument 



4. The one thing that the parent-training specialists 
seemed to want most from the District was 
more role clarification and training. Although a 
training session was offered for parent -training 
specialists before the 1993-94 summer program, 
no training was offered in conjunction with the 
1994-95 program, (p. 18) 



• Using effective questioning strategies in guided 
process reading; 

• Assessing and monitoring reading and writing 
behaviors; and 

• Using predicting/confirming activities and 
mapping techniques that clarify thinking. 

°*Tbc four type* of reading in whale language instruction are: 1) 
Read Aloud, 2) Shared Reading. 3) Guided Process Reading, and 
4) S pent Sustained Reading. These reading type* were included in 
initial teacher training in 1994. 



5. The program also accomplished one of TEA’s 
stated goals— to train teachers so that more 
acceleration could take place in the regular 
classroom, (p. 18) 

6. The overwhelming majority of teachers (92. 8%) 
indicated that they would continue to use the 
methods and materials of the program during 
the regular school year. Teachers said that the 
record-keeping forms and the fact that the 
program methods are based in a whole language 
approach will be useful in the regular class. 

(P 18) 
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Management Responses 



Recomme ndation #1 (Do not concur) 

Funds are better utilized where they directly impact instruction. Principals are responsible for 
programs at their individual campuses. For the 1995-96 Optional Extended Year Program 
campuses will participate on a completely voluntary basis. Campus principals will be required 
attend all planning meetings and assume responsibility for the program at his/her campus or 
delegate responsibility for program coordination to someone at the campus. 

Recom mendation #2 (Concur) 



all 

to 



Principals are responsible for implementation of the program at his/her individual campus. 

atenals and resources will be discussed at the planning meeting for the 1995-96 Optional 
Extended Year Program planning meeting. 



Recommendation #3 (Partial concurrence) 

The need for planning is acknowledged. However, one hour of daily teacher planning time was 
a ocated the second year of the program. It was the principal's responsibility to schedule planning 
time. Principals at individual campuses could have scheduled planning time to meet individual 
campus needs. The Parent Training Specialist was responsible for facilitating parent contact at 
the campus to which he/she was assigned. These issues will be addressed at the 1995-96 Optional 
Extended Year Program planning meeting for principals. 

R ecommendation #4 (Partial concurrence) 



Follow-up planning and discussion is the responsibility of each individual campus and should be 
initiated by the principal. Principals will be reminded of this requirement at the Optional Extended 
Year planning meeting for principals. 



Recommendation #5 (Concur) 

Use of running records will be an optional campus assessment for the 1995-96 Optional Extended 
Year Program. Inservice training will need to be conducted at the campus level as needed. Use 
o Reading Recovery teachers for follow-up on running record training at individual campuses is 



ii 
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Open Letter Concerning Retention Reduction and Acceleration 

The issue of student retention (i.e., requiring a student to repeat the same grade level) has 
been a concern in the Austin Independent School District (AISD) for many years. Since 
1984-85 AISD policy EIE (Local) has permitted the "placement" of elementary students in 
the next grade level in lieu of retention, provided that the placed student receive support 
from an alternative program to meet his/her identified needs. (Being placed is defined as the 
situation that occurs when an elementary student who has not met the requirements for 
passing his/her grade level is put in the next grade with the hope that further intervention 
will help the student catch up with her/his peers.) This option, and other alternatives to 
retention (e.g., transitional first grade and alternative placements) have long been encouraged 
in AISD, based on national and local research on the negative effects of retention for most 
students.’ Studies by the District’s Office of Research and Evaluation (ORE) beginning in 
1981-82 found that retention is not beneficial to most students’ long-term achievement and is 
associated with high dropout risk. (See the "Bibliography" for a list of retention-related 
publications by ORE.) AISD’s retention rates have fluctuated since 1980-81 (the first year 
ORE began tracking retention rates), but over the last six years, retention rates have declined 
in all elementary and middle school grades and in grades K-12 overall. 

Against this backdrop of long-standing District concern with retention, and the long-term 
decline in AISD retention rates, it is difficult to render an unambiguous judgment on the 
effect of the Accelerating Literacy Program (ALP), the program AISD adopted with State 
Retention Reduction grant funds, on the elimination of students’ need to be retained. 
Indeed, information gathered during the first-year evaluation of the program in staff surveys 
and informal interviews indicated that very few of the teachers or principals would have 
retained first-grade students with or without the program. A few teachers also indicated (in 
informal settings) that the policy of the Texas Education Agency (TEA) to regard placements 
as retention, for school evaluation purposes, discourages schools from placing students; 
therefore, they suggested, most students simply get promoted. 

Retention rates aside, however, given that, from the outset, TEA encouraged program 
diversity and experimentation among the districts receiving Retention Reduction funds, 
particularly in the area of alternative assessment, the program seems to have met most of 
its other goals, chiefly, improving students’ foundation for second and third grade, 
developing and using alternative assessment instruments such as portfolios, and training 
teachers to use the "accelerating" methods and materials of the summer program during the 
regular school year. 

Despite District policy and the program’s successes, the issue of student achievement and 
further intervention for students to catch them up with their peers ("acceleration") 
remains. The District’s policy on placement requires an alternative program for^ students 
who “fall two years or more below grade level in reading, mathematics, or both.” The first 
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year ALP evaluation found that no commonly used standard of reading achievement was 

^ n ? 1 5! ed ° n a district ^ ide basis * 10 other words, "performing on grade level" has not been 
defined for each grade in terms of specific performance indicators. This fact suggests that 
teachers have to use their own judgment or school norms to determine what a student should 
know to be promoted, a situation that hardly leads to a systematic norm for student 
achievement. If it is not possible to determine uniformly where a student is performing 
relative to grade level, the District’s policy on placement is undercut. Further, there does 
not seem to be any systematic process for determining the needs of students who require 
placement rather than promotion; neither does there appear to be any systematic provision for 
identifying alternative programs for placed students based on these needs. 

Teachers need all the resources they can get to know what their students have achieved and 
how to propel them forward along a learning continuum that will prepare them for life. 
Retention has been detrimental for many students. Studies have shown that, on the whole 
students who are retained do not catch up with their peers and that they are at greater risk of 
dropping out of school than students who are not retained. Yet simply passing students along 
without any attention to their continuing needs is not helpful either. If teachers are to 
continue placing students, the District needs to ensure that students who need a little more 
help will get the attention they need through intervention programs that support them. 

Indeed, District policy requires that students who have been retained or placed and who still 
fall two years or more below grade level in reading, mathematics, or both will be placed in a 
program designed for rapid progress in language arts, reading, and mathematics (emphasis 



ALP is a possible candidate for that intervention role in language arts/reading. 
(Recommendations for program improvement are listed in the "Executive Summary.") By 
design, the program is adaptable to different student learning styles and achievement levels, 
and evaluation of the summer program indicates that it can be used successfully to improve 
reading skills. ALP is also flexible in its design structure. The District could use this 
program as an after-school program, integrate it into the general curricula, or continue it as a 
summer program. Further study will be required, however, to determine if ALP students’ 
achievement has been accelerated to a level comparable to that of their peers, and to assess 
whether students need additional help to be successjul in the second or third grade. 
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The Accelerating Literacy Program: 1994-95 

Final Report 



Introduction 

Program Rationale 

The Austin Independent School District (AISD) developed its Accelerating Literacy Program 
(ALP) in 1993 to provided additional support for students who were having trouble passing the 
Texas Assessment of Academic Skills (TAAS) in the elementary grades. At the time the program 
was developed, 13 Austin schools were either on the AISD “C” list of schools "in need of 
required improvement" (i.e., less than 25% of students passing all sections of the most recent 
TAAS) or on the Texas Education Agency’s (TEA’s) list of schools that are “clearly 
unacceptable” (less than 20% of students passing all sections of the fourth-grade TAAS). 

"Failure" for these schools and students, many educators would assert, begins much same kinds of 
academic foundation to which some of their peers have had access, and when the schools do not 
bridge the gap in earlier grades between what the students know and what they need to know. 

When schools have been unsuccessful at bridging that gap, teachers and principals traditionally 
have had to choose between passing students who are unprepared for the next grade or retaining 
them. Even with the alternative of "placing" students permitted in AISD policy, an early 
intervention program was needed to "accelerate" the progress of low-achieving students so that 
they could achieve on a par with their peers. (Being placed is defined as the situation that occurs 
when an elementary student who has not met the requirements for passing his/her grade level is 
put in the next grade with the hope that further intervention will help the student catch up with 
her/his peers.) 

Selection of the Accelerating Literacy Program 

Campus and District administrators recognized that the ability to read well is essential to success 
in all other subject areas in which the students needed help. They researched programs designed 
to deliver concentrated reading instruction to a targeted group of “at-risk” students. With an eye 
toward either an extended school day and/or school year, and working through the Region XIII 
Education Service Center, these educators selected the Accelerating Literacy Program (ALP). 
Education Consultant Services (ECS) of Imperial Beach, California offered in-service training for 
teachers and administrators built upon a philosophy of curriculum flexibility based on the needs of 
individual students. ECS’s in-service had grown out of experiments in reading interventions in 
the South Bay (CA) Union School District, a district demographically similar to AISD, and with 
similarly low reading 
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achievement within its “at-risk” populations. Furthermore, the mechanics of the program 
resembled certain intervention procedures used at AISD’s Ortega Elementary School, which once 
had low achievement but has substantially improved its TAAS scores. 

Working with South Bay Union School District personnel, ECS created the Accelerating Literacy 
Program, a reading curriculum characterized by intensive reading instruction, with roots in the 
Reading Recovery/Whole Language Program developed by New Zealand educator. Dr. Marie 
Clay. The training material provided by ECS claims that New Zealand has the highest literacy 
rate in the world. The Accelerating Literacy Program varies from the Reading Recovery model 
in that it is adapted for small classrooms, usually six to eight students, rather than one-to-one 
instruction. The ALP is characterized by the following: 

• Early intervention, 

• Short-term, intense help, 

• Building on strengths, 

• Focus on how-to, 

• Action oriented, 

• Accelerated progress, 

• Reading/writing connection, 

• Focus on meaning, 

• Sound-letter relationship, 

• Flexibility, and 

• Staff development. 

AISD educators also identified parental involvement as an important resource for improving the 
reading achievement of program participants. As District educators began planning the ALP, they 
began identifying methods of integrating parental involvement in the program. A series of 
summer workshops was planned to educate parents on strategies and activities for developing and 
furthering their children’s reading education at home. In addition, a Parent-Student Reading 
Activity Log was sent home with the students. 

Program Funding 

In response to a request for proposals (RFP) issued by TEA’s Division of Accelerated Instruction, 
AISD submitted a proposal for funding the Accelerating Literacy Program in December, 1993. 
The District applied for a grant equaling $298,132 to fund the program for the 1993 -94 school 
year. The grant was approved by TEA at the reduced amount of $223,599 on April 27, 1994. 

On October 1, 1994, AISD submitted an application to renew the grant for the 1994-95 school 
year. The first and second years’ budgets are detailed in Figure 1 . 
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FIGURE 1 

ACCELERATING LITERACY PROGRAM BUDGETS 



•' Ilf . Class/Object 


Hil-94 


1994-95 j 


Payroll Costs 


$126,342 


$175,728 


Purchased and Contracted Services 


16,165 


13,500 


SuDDlies and Materials 


77,577 


30,871 


Other Operating Expenses 


3,515 


3,500 


Total Direct Costs 


$223,599 


$223,599 



Some realignment in the program design was required for the first year because the grant was 
funded late and at a reduced amount. The position of school improvement planner included in the 
application was eliminated. With the exception of two schools, all campuses ran a four-week 
program during the summers of 1994 and 1995. 

Outlook for Program Continuation 

The Accelerating Literacy Program was funded by the Texas Education Agency s Division of 
Accelerated Instruction. The Texas Legislature awarded the District a Retention Reduction grant 
in 1993 and funded a continuation grant in 1994. The funding for the Retention Reduction grant 
has not been continued by the Texas Legislature beyond the 1994-95 school year, although the 
program has an optional extended year. 
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Program Description 



The Accelerating Literacy Program (ALP) as developed by Evaluation Consultant Services (ECS) 
was chosen for implementation in the 1993-94 summer session because of its reported success in 
a comparable environment (i.e., the South Bay (CA) Union School District). The curriculum is 
flexible enough that it can be used as a stand-alone program (such as AISD’s program) or as an 
adjunct to regular school year reading instruction. 

The thematic principle upon which the program operates is that of integrating reading, writing, 
listening, and speaking activities into a whole language approach to instruction. The program 
seeks to accelerate the learning process by building on students’ successes. According to the 
theory of the program, when a teacher determines that a student has mastered text at a given 
level, the teacher should immediately challenge the student at a higher level. According to the 
ALP instruction manual, this objective is pursued through the following steps: 

• Evaluating the reading process through miscue analysis; 

• Applying the four types of reading 1 in whole language instruction; 

• Using effective questioning strategies in guided process reading; 

• Assessing and monitoring reading and writing behaviors; and 

• Using predicting/confirming activities and mapping techniques that clarify thinking. 

The 1 3 campuses involved in the Accelerating Literacy Program were selected on the basis of 
poor performance by their fourth-grade TAAS takers. Each campus, and in fact each teacher, 
was allowed great latitude in designing a course of instruction to meet the needs of the students. 
The level of coordination between classes varied from campus to campus, although in most cases 
periodic staff meetings were held. During these meetings, the lead teacher could facilitate 
strategy sessions, answer questions, and distribute materials. The frequency of these meetings 
was determined by the staff. 

Staffing 



Five lead teachers were each assigned responsibility for two campuses; one lead teacher was 
responsible for three campuses. Each school had one parent-training specialist (PTS) for the 
program. Most of the schools had four teachers; one school had five teachers, and two 
others ended the program with three because of lack of attending students. The staff seemed very 



'The four types of reading in whole language instruction are: 1 ) Read Aloud. 2) Shared Reading, 3) Guided Process 
Reading, and 4) Silent Sustained Reading. These reading types were included in the initial training in 1994. 
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committed to the program and their students. A few of the teachers and PTSs went out of their 
way to pick up students on their way to school and dropped them off in the afternoon. 

One of the District’s Chapter 1 coordinators acted as the ALP coordinator, although that position 
was not included in the program budget. As a result, her time was somewhat stretched by the 
requirements of managing a program that took place on 13 campuses besides her responsibilities 
as Chapter 1 coordinator. The program coordinator kept up with the program deadlines, planned 
the training workshops, planned and facilitated information meetings, and performed 
miscellaneous other duties related to the program as a whole. The individual campus programs 
were the responsibility of the principal. 

Training 

In April 1994, trainers from ECS conducted a three-day, 15-hour workshop on the Accelerating 
Literacy Program curriculum for AISD teachers and administrators, most of whom participated in 
the 1994 summer program. Release time was given to participating personnel. Among the main 
features of the program were lesson planning, diagnosis of particular problems by charting 
running records, miscue analysis training, and information on books and materials. 

A second 15-hour workshop was conducted by a consultant from ECS on June 20-21, 1994 for a 
group of kindergarten and second-grade teachers who would be utilizing some of the ALP 
techniques in their classrooms during the 1994-95 school year. Training the second group of 
teachers was intended to help ensure continuity in student learning and support. A brief 
questionnaire, developed by the ORE, was given at the conclusion of the second workshop. 
Responses to the questionnaire reflected general satisfaction with and enthusiasm for the program. 

A few of the teachers did indicate that they felt that the amount of material covered by this 
workshop was too much to be absorbed in two days. 

After the data were collected from the first summer session, it was determined that the teachers 
needed more training on conducting the running records assessment. In October 1994, 12 hours 
of training specifically on the running records was conducted by AISD staff involved in the 
Reading Recovery program. Four hours of training were dedicated to general lesson planning 
topics. In April 1995, two eight-hour sessions were conducted, one on the reading aspects of the 
curriculum and the other on the writing curriculum. 

Program Resources 

There were great disparities between the resources available at different campuses. Through a 
different grant, some campuses had computers, complete with technicians, available. Some schools 
did not even have access to telephones and copiers. Since the program took place during the 
summer. District services were greatly diminished; e.g., school mail was not distributed on a daily 

basis. 
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Population Served 



Figure 2 illustrates ALP’s sex and ethnic 
distribution during each of the program 
years. Students were selected for the 
program based on teacher 
recommendation. The selection criterion 
was that all students in the program 
should be in danger of being placed or 
retained for the next academic year. As 
Figure 3 shows, the vast majority of 
students in the program had been 
promoted at the end of the academic 
year. 2 In fact, of 
372 ALP students in the 1994-95 
program, only 70 (19%) were 
recommended to be placed or retained. 

In 1994-95, the program expanded 
officially to include second graders. 
Sixty-nine students who were in the 

1993- 94 program were recom- 
mended for and enrolled in the 

1 994- 95 program. At the end of 
the school year, none of these 
students had been recommended 
for retention, and slightly fewer of 
them (15.9%) had been 
recommended for placement. 



[H Promoted g Placed g Retained 



FIGURE 3 



ALP PLACED, PROMOTION, AND RETENTION 
STATUS AT THE BEGINNING OF THE PROGRAM 




1993-94 1994-9S 



FIGURE 2 

ACCELERATING LITERACY PROGRAM 
DEMOGRAPHIC SUMMARY 



SEX 


1993-94 


1994-95 




Male 


48% 


53% 


Female 


52% 


47% 


ETHNICITY 


Hispanic 


53% 


57% 


African American 


44% 


38% 



2 Only 295 of the participants are included in the 1993-94 statistic. Seventy-two students are not included because of 
inability to obtain correct student identification numbers. 
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Evaluation Overview 

In 1993-94, AISD’s Office of Research and Evaluation (ORE) was assigned the responsibility of 
evaluating the program chosen for the Retention Reduction grant. An evaluation consultant, 
reporting to ORE’s senior evaluator, was contracted to fulfill the design and execution of the 
evaluation (see ORE Publication Number 93.24). The second-year evaluation begun by ORE 
continued under the Department of Performance Audit and Evaluation (A&E), which was formed 
in January 1995. Another evaluation contractor was employed to conduct the 1994-95 evaluation. 

Program Revisions 

Several changes took place between the submission of the grant proposal and its subsequent 
completion. Program objectives and evaluation questions were changed (see below), the evaluation 
contractor changed between the two summer programs, and the program structure changed on 
some campuses. 

The program objectives changed from the submission date to the implementation date of the first 
year. In the original proposal submitted to the Texas Education Agency, four major program 
objectives were defined: 

1 . All students who participated in the Accelerating Literacy Program will avoid the need to be 
retained; 

2. Students will have a stronger foundation for second grade, as shown by at least a four-level 
gain on the diagnostic running record in reading; 

3. Parents will become partners in the educational process as shown by their participation in 
workshops, which stress home teaching strategies and the development of homemade 
materials and learning activities; and 

4. A school improvement planner will continue to expand literacy intervention efforts in these 
schools. 

The program objectives for the 1993-94 school year were amended as follows: 

Objective #2 was changed to read: 

Students will have a stronger foundation for second grade, as shown by at least a two-level gam 
on the diagnostic running record in reading. 

Objective #4 was eliminated. 

The program was not completely funded and reductions had to be made. It was decided that 
this position could be eliminated without substantially reducing the program impact. 

Objective #1 was problematic because of District policies discouraging retention. Information 
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gathered the first year of the program in staff surveys and informal interviews indicated that very 
few of the teachers and principals would have retained first-grade students with or without the 
program. For the second year of the program, the wording of the objective was changed so that 
students who would have been placed were selected for the program, as well as students who 
would have been retained. The change simply clarified the selection process. 

Objective #2 was changed from a four-level gain to a two-level gain based on a clearer 
understanding of the reading levels determined through the use of the running record form (see 
"Performance Indicators" below). 

The change in the evaluation contractor occurred on October 20, 1994. The new contractor had 
less than a month to acquaint herself with the program and to take over the evaluation before the 
first intersession program began November 17, 1994. Information and assistance were provided by 
the former contractor, the senior evaluator, and the program coordinator to help orient her quickly. 

The program structure changed on some campuses. After working with the program for one 
summer, the staff at three schools decided that an important distinction between this program and 
the regular school year needed to be made. They believed that the students should not be made to 
feel as if they were losing their summer break. These schools changed their program schedule from 
a five-day, four-hour program to a four-day, 4 '/ 2 -hour program. The students brought a sack lunch 
to eat in the late morning. Two other schools became year-round schools; therefore, they 
implemented the program in two two-week sessions while the schools were having intersession 
break. 

The Evaluation Questions 

The 1994-95 evaluation proceeded with these program revisions in mind. A basic set of evaluation 
questions was developed to determine the effectiveness of the program based on the objectives, as 
well as on other District objectives. The 1994-95 evaluation questions were as follows: 

1 . Do students in the Accelerating Literacy Program make significant achievement gains? Are 
the gains made by students in this program greater than they would be in other AISD school 
programs? Are these gains sustained over a period of time? 

2. Is the Accelerating Literacy Program different from other classroom interventions being 
implemented in the District? If so, how? If not, to what other intervention(s) is it similar? 

3. Are parents being encouraged to be involved in the program? What kind of parent 
involvement is being encouraged? What methods are being used to encourage parent 
involvement? How effective are these methods? 

4. Are teachers being effectively trained and supported to use any unfamiliar teaching strategies 
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involved in the use of acceleration methods? 

5. Are teachers trained and able to develop and use portfolios for student assessment? Were 
student portfolios effectively used for student assessment? Were they used in 
student/parent/teacher communication about student achievement? 

6. Is the Accelerating Literacy Program a cost-effective program worthy of continuing and/or 
expanding? 

The 1994-95 evaluation is limited in that the evaluation design did not afford a 
determination of how much of the developmental progress made by the students in the 
program would have been made without the program. A control group design for comparing 
the performance of program students to that of an equilavent group of nonprogram students would 
normally be employed for this purpose. Since standardized test scores were barred from the 
evaluation by TEA, however, the evaluation could not avail itself of the extensive test information 
on hand for comparing student performance. Instead, the same alternative assessments by which 
the progress of program students were measured would have had to be carried out on a group of 
comparable nonprogram students. Since some of these measures required extensive teacher 
training, a control group design was not practical. Therefore, while the evaluation could determine 
whether progress was made by program students, the evaluation could not rule out the possibility 
that the same amount or more progress would have been made by these students using another 
program or no program at all. Nonetheless, a range of performance indicators were employed in 
the evaluation, as described below. 

Performance Indicators 

Good performance indicators are well aligned with the program objectives. Those indicators 
should provide the foundation for the assessment tools used to evaluate the program. Since 
standardized tests were against the philosophy of the program and not allowed by the grant, 
alternative assessments were required to determine the effectiveness of the program. Alternative 
assessment provides a different type of information from that provided by standardized tests. 

Often, alternative assessments can provide everyone, from students and parents to the 
superintendent, with actionable information about what the children know related to what we say 
they need to know. Some standardization, however, is necessary for any assessment to be useful 
on the level of programmatic evaluation. The standardization required is: 

1) A standard of expectation for student success, 

2) A standard definition of the curriculum mastery objectives and indicators, and 

3) A standard training program for teachers on what the indicators mean and how to rate 
them. 

These three tenets can provide the foundation for a system of cross-checking teacher assessment 
and program performance. An often-expressed concern is that teachers will inflate student 
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achievement on the performance indicators used for standardization in alternative assessment. 
Portfolios should support teachers’ assessments by including developmental demonstrations of the 
students’ progress along a continuum of achievement. Also, a system of cross-checking can be 
used to verify teachers’ assessments. By using other teachers in the program, or the lead teachers, 

assessments can be checked to make sure that expectations and objectives are consistent across 
the program. 

Since alternative assessment is not standard practice throughout AISD, the necessity of using such 
assessments required the creation and/or adaptation of several assessment instruments, some of 
which were unfamiliar to both the program and evaluation staff. These assessment tools were: 

The Portfolio Assessment Form . At a minimum, portfolio assessment is simply compiling samples 
of the students work, which represents their progress along a learning continuum. At best, a 
portfolio teaches young people to take responsibility for their own learning by providing 
conceptual structures for self-reflection. While it is not necessary that students and parents be 
involved in the process, research has shown that they can be a great enhancement to the analysis 
process. By involving students in the process, teachers can empower students to take charge of 
their own education by giving them the tools to evaluate their work and measure their progress. 

A portfolio also provides parents with concrete evidence of their child’s progress in real terms, 
instead of constructed generalizations (e.g., letter grades). Involving students and parents in 
portfolio assessment is a lot of work for everyone involved, but it promotes a relationship among 
parents, teachers, and students that says, “All of our opinions about the students’ progress are 
valuable.” 



The most important thing the program must do to make portfolio assessment successful is to 
provide teachers and campus administrators with the necessary resources to do their job well. 
Explicit definitions of the program’s expectations of student learning detailed by grade, and how 
that learning is to be evaluated, are absolutely required. It is impossible to do a systemwide 
assessment of portfolios without systemwide definitions of assessment criteria and indicators. 

The next step is definition of the portfolio itself. There are many different types of portfolios and 
equally as many opinions as to which is best to use in portfolio assessment. One of the most 
difficult processes in choosing the type of portfolios that will be developed is often deciding what 
will go into the portfolio. One option is to select work that demonstrates important educational 
milestones or progress. This type of portfolio can be the most difficult to develop, but it offers 
students an opportunity to identify their own milestones and developments, which can help the 
students take ownership of their educational development. 

The Portfolio Assessment Form used for ALP was designed to be completed by the teachers. The 
form is simply a checklist of the assessment tools which were to be included in the portfolio and 
an inquiry about the students’ progress and the portfolio’s use. 
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The Acceleration literacy Program Developmental Checklist To measure student progress 
during the program, teachers were asked to rate each student’s development in the areas o 
reading, writing, and listening/speaking using a list of 26 learning indicators at the beginning and 
end of the program. The Development Checklist was adapted from several sources and based on 
the developmental indicators teachers said that they looked for in their classrooms. 

Teachers used a five-part scale to rate each student’s abilities in writing (13 indicators), reading 
(seven indicators), and listening/speaking (six indicators). The rating scale was defined as 

follows. 

• Student will be considered to demonstrate a skill or behavior consistently if s/he is able to 
demonstrate it 85 to 100 percent of the time; 

• Student will be considered to demonstrate a skill or behavior most of the time if s/he is 
able to demonstrate it 55 to 85 percent of the time; 

• Student will be considered to demonstrate a skill or behavior occasionally if s/he is able to 
demonstrate it 15 to 55 percent of the time; 

• Student will be considered to demonstrate a skill or behavior not yet if s/he is able to 
demonstrate it less than 15 percent of the time; and 

• Not Rated should be used if the skill or behavior was not observed. 

For analytic purposes the scale was converted to a numeric format, in which 

\ = Not Yet, 2 = Occasionally, 3 = Most of the Time, and 4 = Consistently. 

When “Not Rated” was selected, or the teacher’s response was ambiguous or nonexistent, the 
item was not used in calculations. Comparison of the pre- and postprogram assessment was used 
to measure students’ general and specific development in reading, writing, and communication. 

The Running Record Form . This form, borrowed from Reading Recovery, was intended as a 
more specific indicator of student progress in reading accuracy and self-correction abilities. The 
teacher checks off each word the student reads correctly and notes the type of miscues, or errors, 
the student makes on incorrectly identified or pronounced words. After the reading the teacher 
records the number of “running words” in the text and the number of miscues the student made. 
The teacher then calculates the accuracy rate (Number of Correct Words/ Total Running Words) 
and the self-correction rate (Self-Corrected Error/Total Errors). Based on this information the 
teacher indicates the text level of the story in relation to the student’s reading ability (i.e.. Easy, 
Instructional, Hard). Comparison of the pre- and postprogram assessment was used to measure 

students’ reading development. 
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I h . e Read-Retell Worksheet . The Read-Retell activity was to be conducted at the beginning and 
end of the program using the same story. Teachers were instructed to read the story to their 
students twice or three times if necessary for the students to get meaning from the story but not to 
memorize it. An assessment form, which asked how many words were used and how clearly and 
effectively the students retold the story, was provided for each activity. Comparison of the pre- 
and postprogram assessment was used to measure students’ development in writing and 
comprehension. 

I_ he Audiotape Form . The teacher was to have each student make an audiotape of her/his 
reading. The audiotape should have included two examples of the student’s reading, one at the 
beginning of the program and one at the end of the program. The readings did not need to be the 
same text. Teachers completed one audiotape form per student to state how much progress was 
demonstrated on the tape and whether the tape was a true reflection of the student’s progress. 

S t udent Interviews . This interview was to be conducted during the last week of the program. 
Teachers were instructed to be sure students understood that saying that they did not like to 
read/write was perfectly all right, and they were not to push students for "appropriate" answers. 
Teachers were also instructed to use whatever terminology was familiar to the student. The 
purpose of the interview was to determine if the students had literature interaction outside school 
and of what that interaction consisted. 

Parent-Child Reading Activity Log . This form was to be distributed to students on the first day of 
the program to take home to their parent(s). Parents were to send updated logs to class with their 
child on Fridays. Parent Training Specialists were to explain the purpose of the log to the parents 
and follow up with any family which needed help in completing the log and/or other activities. 

The log was used as a record of parent-child daily reading activity. 

P arent and Staff Surveys . The parent surveys were sent home with the students about a week 
before the end of the program. Parents were to complete the surveys and return them with the 
students before the last day of the program. Staff surveys were distributed during the last week 
of the program. (Appendices A and B detail the results of the staff surveys.) 

T eachers Anecdotal Notes . These notes are very informal observations of the student’s daily 
progress in the program. Teachers were asked to record observations as often as possible, but at 
least once a week. The notes helped the evaluation consultant to be aware of how the students 
and the teachers were interacting with each other and with the work required by the program. 

Informal Interviews with Program Staff . These were simple conversations with various members 
of the program staff and were totally unsystematic. They ranged in length from a very few 
minutes to an hour or more. 
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Findings 

Elimination of Students’ Need to be Retained 

The elimination of students’ need to be placed or retained is not currently a measurable 
objective. First, the program's 1994-95 selection criterion, that all students in the program 
should be in danger of being placed or retained for the next academic year, lacks specificity and 
depends, in turn, on a District policy for promoting, placing, and retaining students which itself 
needs additional definition. AISD policy states that “a student may be placed in the next grade 
level if the student is achieving to his or her maximum ability, subject to" certain requirements: 

1 The student is placed in an alternative program which will meet her/his identified needs; 

2. Unless a student is only receiving speech/language services, the ARD (Admission, Review, 
and Dismissal) Committee will make the placement decision for all special education 
students; and 

3 Students who have been retained or placed and who still fall two years or more below 
grade level in reading, mathematics, or both will be placed in a program designed for rapid 
progress in language arts, reading, and mathematics. 

A method for identifying student needs is not described, nor is the measure of what constitutes 
"below grade level" performance identified. Although the state-defined essential elements 
provide a general framework for what students ought to learn at each grade level, the District 
lacks specific performance indicators for each grade and subject area. In reading, the 1993-94 
ALP evaluation found "no common set of reading achievement standards." Therefore, the 
program's objective of having all students out of danger of being placed or retained (i.e., 
performing at or above grade level) cannot be met until performance requirements are defined. 
The absence of districtwide definition has meant that teachers are forced to use their own 
discretion or school norms to determine what a student should know to be promoted-or, 
conversely, placed or retained (the program's identification criterion). 

Second, AISD has discouraged the retention of elementary students for many years, and high 
percentages of students were promoted each year even before the advent of ALP. Although ALP 
teachers were able to supply reasons that students needed to be placed or retained, in 1993-94 
and 1994-95, respectively, 80.7% and 81.2% were promoted. A few teachers also indicated (in 
informal settings) that the TEA’s policy to regard placements as retention, for school evaluation 
purposes, discourages many schools from placing students; therefore, they suggested, most 
students simply get promoted. 
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Improve Students’ Foundation for Second/Third Grade 

Reading 

Running Records. Many problems with the first-year implementation of the running records were 
not completely eradicated in the second year. In spite of more than 12 hours of training on 
implementing the running record assessments, many teachers did not include enough information 
on the data collection forms to make the data useful. Overall, only 155 student records were 
usable out of 372 students (41.67%). 

In addition to the number of forms that were unusable, 34.8% of the usable records showed 
no change m text level. Although it is likely that “no change” is accurate in some of these 
54 records, whole classes showed no change in some cases. Also, because of the way the 
initial assessment was conducted in a few cases, the accuracy of the initial reading level of 
those students is questionable. Better communication between the evaluation consultant and 

some of the teachers as to how the assessment was to be conducted might have made the data 
collection more accurate. 



FIGURE 4 

AVERAGE CHANGE IN READING 
TEXT-LEVEL - RUNNING RECORDS 
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In spite of these problems, the usable data indicated that the students gained more than the 
two text-level achievement gain 
that the program predicted. 

Overall, the running records 
indicated that ALP students 
increased their reading text level 
an average of 2.93 reading levels. 

First graders improved even more 
than second and third graders in the 
program, with an average increase of 
3.21 reading levels on the running 
records assessment (see Figure 4). 

Developmental Checklist . Analysis 
of the ALP Developmental 
Checklist of Learning Indicators 
indicated that ALP students made 
statistically significant gains in 
their overall reading skills for the 

second consecutive year. Students participating in the 1994-95 Accelerating Literacy Program 
gained an average 0.48 levels in reading skills (based on a four-point scale, where “4” is the 
highest possible rating). See Figure 5. Students in the 1993-94 ALP had average reading skill 
gains of 0.47 on the Developmental Checklist. 
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FIGURE 5 

Accelerating Literacy Program Developemental Checklist 



Interestingly, the second 
graders performed best 
on this assessment, 
although the first 
graders averaged greater 
increases in achievement 
on the running record 
assessment. Possible 
reasons for the 
difference between the 
results of the two 
assessment tools 
include: 1) The 
Running Record 
assesses either different 
skills or only some of 

the skills reflected in the checklist; and 2) All of the records for the Developmental Checklist 
were usable; therefore, more accurate information was gathered by that assessment than for the 
Running Records where only 41 .67% of the records were usable. If the two assessments continue 
to be used, and assessment information from the Running Record is improved, the first reason 

could be eliminated as a possibility. 
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FIGURE 6 

Oral Reading Progress 
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Audiotape Form . Analysis of the 
Audiotape Form shows that the majority 
(56.1%) of ALP students made “good” 
or “great” progress in their oral reading 
during the 1994-95 program. On the 
assessment form, teachers indicated that 
the audiotape records were 96% accurate 
reflections of the student reading 
progress during the program. 

Teachers recorded greater progress for 
the first graders on this assessment, once 
again. Of first graders, 62.2% made 
good or great progress. 



Teacher Survey Of the 28 teachers who responded to the Teacher Survey, 100% indicated that, 
overall, the program was very effective or somewhat effective in improving the students’ ability to 
construct meaning in reading activities. 



O 

ERIC 



15 



22 



94.13 



Accelerating Literacy Program: 1 994-95 



Writing and Listening/Comprehension 



R ead-Retell ALP students averaged a one-level increase in their writing level based on the ALP 
Scale of Writing Development adapted from The Whole Language Catalogue: Supplement on 
Authentic Assessment 3 Overall, students used 29. 1 more words to retell the story in the 
postprogram assessment. Teachers indicated that 20,5% of the time students were more able to 
read their own retelling of the story. The students’ stories were more likely to make sense 40.7% 
of the time, and it was more likely that the story had the same conceptual meaning as the original 
story 38.7% of the time. 

This assessment was not available for the two schools, Metz and Winn, which did two-week 
intersessions rather than the summer program. The assessment results are based on analysis of 
1 68 records. 



Developmental Checklist In addition to 
reading, the ALP students also made 
substantial progress in both writing and 
communicating during the program. 

Analysis of the writing and listening/speaking 
scores indicates that the progress made in 
these areas (.32 and .40 incremental changes 
based on a four-point scale) was not as great 
as was made in reading (.48); however, 
according to the philosophy of the program, 
the improvement is assumed to be interrelated. 
If the gains made in writing and 
listening/speaking did contribute to the gains 
in reading, the converse may also true: as the 
students continue to improve their reading 
skills, their skills in both written and oral 
communication should improve. 



FIGURE 7 



ALP DEVELOPMENTAL CHECKLIST 
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Once again, the Developmental Checklist was not designed with third graders in mind; therefore, 
they would begin the program already very near the highest performance indicators included on ’ 
the checklist. This higher preprogram level may account for the lower average change seen in the 
writing and listening/speaking skills. 



Teacher Survey . Of the 28 teachers who responded to the Teacher Survey, 100% indicated that 
the program was Very effective ’ (52.4%) or “somewhat effective” (47.6%) in improving the 
students’ ability to construct meaning in writing activities. 



3 See Appendix C. 
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Interest in Literacy 

Teacher Survey Of the 28 teachers surveyed, 95.2% indicated that their students were “very 
likely” or “somewhat likely” to choose to participate in literacy activities in their free time more 
often after the program. When asked if the program helped their students to become more self- 
guided in their education, 89.5% said that it had. The majority of teachers who went on to 
explain their answers said that students volunteered to engage in reading or writing activities 
when they finished their work. Many teachers also said that the students worked longer and/or 
more successfully at self-correcting before asking for help. 

Student Interviews . Teachers conducted student interviews at the end of the program to get 
information on what students were feeling about reading and writing. Students indicated, overall, 
that they enjoyed reading and writing. Only six students (1.6%) indicated that they did not like to 
read, and 16 (4.3%) indicated that they did not like to write. The reasons students gave for liking 
to read and write were encouraging. Students said that they like to learn , that reading and 
writing was fun, that it made them proud of themselves, and that learning to read and write gave 
them hope for their future. 

Increase Parent Involvement 

Very few of the teachers indicated that the parents had seen their child’s portfolio. Although 
38.1 % of the respondents to the teacher survey indicated that they had less contact with parents 
during the program than they normally would, 42.9% indicated that they had at least a little bit 
more contact with parents. Of the 372 students who participated in the program, 130 (34.9%) 
returned reading logs to indicate that a parent was reading with the student. According to the 
teachers, some parents did read to the children but did not turn in the reading log for the last 
week 

Most (87.5%) parent-training specialists (PTSs) indicated that they held parent meetings before 
and during the summer program. Although few of the parents indicated that they attended these 
meetings, one PTS reported that she had offered to provide transportation for parents. Other 
types of parent involvement that were encouraged included classroom visitations, acting as 
teacher aides, and working one-on-one with their child(ren) at home. PTSs contacted parents by 
telephone, letters, and home visits. Some PTSs sent pamphlets home on positive parenting topics. 

Many PTSs and lead teachers indicated that there was no difference between the methods of 
contacting parents and encouraging parent involvement in the program and during the regular 
school year. Others said that this program was better because staff had daily contact with parents. 
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Overall Program Success 

Overall, the program seems to have met most of its goals. All of the teachers surveyed 
indicate that the students made “somewhat more” (45%) or “much more” (55%) progress than 
they would have in a typical four- week period during the regular school year. Teachers were 
generally pleased with the training that they received, and the portfolios were used successfully in 
the program. Teachers reported 96.3% of the time that portfolios were accurate representations 
of the students’ achievement during the program. The teachers also indicated, in some cases, 
where portfolios were not representative of student achievement that it was because the students 
were excessively absent. 

The one thing that the parent-training specialists seemed to want most from the District 
was more role clarification and training. Whereas teachers and lead teachers had some specific 
role definition, the program coordinator left the PTS’s role to be defined by the campus staff. The 
program coordinator felt strongly that the role clarification should come from the campus level so 
that PTSs could be used as each individual campus needed. PTSs indicated that the fact that there 
had been little definition of their role during either the school year or the program put them in the 
position of defining their own roles. Some PTSs said that they had not been offered any training 
for the job during the regular school year. Although a training session was offered for parent- 
training specialists before the 1993-94 summer program, no training was offered in conjunction 
with the 1994-95 program. 

The parent-training specialists were the resource most often identified by the teachers 
necessary to making the program effective (38.9%). Other important resources were the lead 
teacher, other teachers who had prior experience in the program, and the training workshops. 

Lead teachers said that all of the resources and materials for the program were necessary to make 
it a success. The Portfolio Assessment Worksheet, the instructional materials, and training were 
mentioned specifically. 

The program also accomplished one of TEA’S stated goals--to train teachers so that more 
acceleration could take place in the regular classroom. With the exception of two campuses, 
Winn and Metz, all of the program teachers attended training workshops. The response to these 
workshops was very positive. On a postworkshop survey, the vast majority of participants 
(91.9%) “agreed” or “strongly agreed” that the workshop content was relevant and useful even 
though there was an even mix of participants who were and who were not familiar with the 
material. The comments about the sessions included high praise for the presenter and 
recommendations that more teachers be offered these workshops. 

The overwhelming majority of teachers (92.8%) indicated that they would continue to use 
the methods and materials of the program during the regular school year. Teachers said 
that the record-keeping forms and the fact that the program methods are based in a whole 
language approach will be useful in the regular class. Although it seems that teachers need more 
opportunity for supervised use of the assessment tools, until they become more comfortable with 
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the techniques, teachers indicated that having written records of student progress was beneficial. 
Some teachers said that being able to keep track of small gains using the portfolio was also 
beneficial to students who needed a boost in their self-esteem. Some teachers were concerned 
about the time it would take to conduct pre- and postprogram assessments in the regular 
classroom. One teacher suggested that classroom volunteers would be necessary to facilitate the 
program during the regular class day. 

Campus-based program managers were not the norm during the regular school year; as a 
result some of the preprogram planning was not adequately handled. Although notices were 
sent to all the schools informing them of funding for supplies and materials and recommended 
deadlines for ordering, several campuses had trouble receiving supplies before the start of the 
program. In some cases this was because they did not order their supplies before the program 
planning period started. In other cases, it was simply a matter of slow delivery or the supplies 
waiting at the warehouse. 

Finally, some teachers, lead teachers and parent-training specialists indicated either in 
informal discussions or on the survey that they felt that many of the students needed more 
time to be successful in the program. Some staff indicated that they could have developed a 
better program if they had students for four longer days and used the fifth day for planning, and 
making parent contact. (Two campuses did hold class four days a week; however, extra planning 
time was not budgeted.) 
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Cost-Effectiveness of the Program 

The Accelerating Literacy Program was almost entirely grant-funded for 1993-94 and 
1994-95. (AISD made in-kind contributions of buildings, utilities, janitorial services, etc.) 
This year marks the aid of the program funding, at least in this form. If the program is to 
continue, funds must be sought from another source. The cost-effectiveness of the program 
should be considered to determine whether to seek further funding for this program. 

The cost of the Accelerating Literacy Program can be examined in a number of different 
ways. One way is the cost per student, obtained by dividing the program budget by the 
number of students served. 



Program budget = $223.599 = $755.40 

# students served 296 per student 

Another way to express program cost is the cost per student per hour of the program, or cost 
per contact hour, which is calculated by dividing the cost per student by the number of 
classroom hours allotted for the program. 



Cost per student 


= 


$755.40 


= $9.44 


Hours of the program 




80 


per student- 
program hour 



The cost-effectiveness of the program may be expressed as the cost per student per unit of 
effect, calculated by dividing the cost per student by the average increase in reading skill 
level. 



Cost per Student = $755.40 = $1,573.75 

Effect on reading skill 0.48 per student per 

unit of increase in 
reading achievement 

Cost-effectiveness may also be expressed as the total cost for all students per unit of effect, 
which can be computed as the ratio of program budget divided by the average increase in 
reading skill level. 4 



Program budget 


= $223.599 


= $465,831 


Effect on reading skill 


0.48 


unit increase in reading 
achievement by all students 



The 1993-94 evaluation report incorrectly reported the cost-effect ratio (total program budget/mean 
gain(loss) in reading achievement levels) as $47,574. The correct ratio is $223,599/0.47= $475,743. 
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This cost-effectiveness measure can also be determined by multiplying the per-student cost per 
unit of increase in reading achievement by the number of students served by the program. 

$1,573.75 x 296 = $465,830 

per student per unit # of students unit increase in 

of increase in reading served reading achievement 

achievement by all students 

Finally, program cost-effectiveness may be expressed as the cost per student per program 
hour per unit of effect , which is calculated by dividing the cost per student per program hour 
by the average increase in reading level. 



Cost per stud ent-program hour = $9.44 
Effect on reading skill 0.48 



Of the three cost-effectiveness measures, the first (cost per student per unit of effect) might be 
the most useful. In other words, for each student served to attain an increase of 1.0 in 
rending level as measured by the Developmental Checklist, it would cost $1,574 per 
student. Using this measure, the 1994-95 ALP was less cost-effective than the 1993-94 ALP 
($1,574 versus $1,296, respectively); however, using the total cost for all students per unit of 
effect from each year, the 1994-95 program was more cost-effective than the 1993-94 program 
($465,831 versus $475,743). The per-student contact hour costs of the two years’ programs 
were roughly equal»$9.50 in 1993-94 and $9.44 in 1994-95. Because the same amount was 
budgeted each year for the program, and program effectiveness was almost identical (0.47 and 
0.48 gains in reading level), the differences in the costs between the years are attributable to 
the number of students served and the number of program contact hours. In 1993-94, 367 
students were served, 71 more students than in 1994-95. In 1994-95, however, students 
received more program contact — a total of 23 ,680 contact hours, compared with 23 ,520 in 
1993-94. The more students served by a program, the less expensive the program will be, 
but the fewer the number of contact hours, the more expensive the program will be, as 
indicated in the 1993-94 ALP report. These offsetting differences between the two years’ 
programs result in the approximately equivalent contact hour costs, indicating that the 
programs were about the same in cost-effectiveness. 

However they are expressed, it should be noted that program costs represent expenditures 
above what the District already spends on a per-student basis. The District should consider 
whether other options are available by which the same gains could be accomplished. Other 
benefits of the program, such as training teachers to use the whole language and portfolio 
methods in their regular classrooms, should also be factored into any program funding 
decision. 



$19.67 

per student per program 
hour per unit of increase 
in reading achievement 
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Accelerating Literacy Program (ALP) - Summer 1995 
Teacher Survey 



At the beginning of the program, what skills needed to be accelerated which would have made 
it appropriate to ‘place’ or retain the Accelerating Literacy Program (ALP) students? 



TEACHERS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Lack of comprehension. 


3.03% 


1 


Students need more exposure to literature. 


6.06 


2 


Students’ inability to sequence stories. 


9.09 


3 


Students need to work on reading strategies. 


15.15 


5 


Students need to work on Phonics. 


21.21 


7 


Students need to work on reading and or writing skills. , 


27.27 


9 


Students need to work on fluency in their reading . 


6.06 


2 


Low self-esteem. 


6.06 


2 


Students aren’t taking instruction well enough to be successful. 


3.03 


1 


1 don’t know. 


3.03 


1 



How did you use ALP students' strength for working on the areas in which the students 
needed acceleration? (Please give an example or two) 



TEACHERS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Used prior knowledge of stories. 


4.55% 


1 


Built on familiar word/word chunks. 


9.09 


2 


Reminded students about using strategies they had already mastered. 


9.09 


2 


Stronger readers read to weaker readers. 


9.09 


2 


Listened to students read every day. 


13.64 


3 


Students chose their own writing topics. 


9.09 


2 


Students made their own dictionaries. 


4.55 


1 


Used peer reading. 


4.55 


1 


Other. 


36.36 


8 
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How did you use portfolios to assess student achievement? 



TEACHERS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Compared beginning to ending achievement levels. 


53.6% 


15 


Referred to daily logs. 


14.3 


4 


Running records, in particular, helped keep track of specific 
strengths/weaknesses. 


25.0 


7 


Built up students self-esteem by showing the students their progress. 


3.6 


1 


Not much. 


3.6 


1 



Overall, how effective was the program at improving ALP students’ ability to construct 
meaning in reading activities? 



Very effective 


Somewhat effective 


Somewhat 

ineffective 


Not effective 


38.1% (8) 


61.9% (13) 


0.0 


0.0 



Overall, how effective was the program at improving ALP students’ ability to construct 
meaning in writing activities? 



Very effective 


Somewhat effective 


Somewhat 

ineffective 


Not effective 


52.4% (11) 


47.6% (10) 


0.0 


0.0 



Overall, are the ALP students in your class more likely to choose to participate in literacy 
activities in their free time than they were at the beginning of the program? 



Very likely 


Somewhat likely 


Somewhat unlikely 


Not likely 


42.9% (9) 


52.4% (11) 


4.8% (1) 


0.0 
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Did the program help students in your class become more self-guided in their education? 
(Please explain.) 



TEACHERS’ RESPONSES 


PERCENT 


NUMBERS 


Yes 


89.5% 


17 


No 


10.5% 


1 2 



EXPLANATIONS: 

Students volunteered more often to read/write independently. 
Students compare home reading with school reading. 

Student shared reading materials from home with the class. 
Students made more successful attempts to self-correct. 

(No) My students were ADD. 



8. How successful was this program at accelerating student achievement, relative to gains made 
by students at similar ability levels over a four-week period in a regular class? (Please 
explain.) 



Very successful 


Somewhat 

successful 


Somewhat unsuccessful 


Not successful 


55% (11) 


45% (9) 


0.0 


0.0 



EXPLANATIONS: 

It was helpful for these students to have materials available during the summer. 
The size of the class made it more realistic to offer individual attention. 

The emphasis on literacy helped focus the work. 



9. How much contact with parents did you have compared to what you would normally have 
had during your regular school year? 



Much more 


Somewhat more 


A little 
more 


No more 


Less 


0.0 


19.0% (4) 


23.8% (5) 


19.0% (4) 


38.1% (8) 
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How often were portfolios used in discussions with parents about student achievement? 



Always 


Often 


Occasionally 


Rarely 


Never 


0.0 


0.0 


10.0% (2) 


40.0% (8) 


50.0% (10) 



How successful was the program at meeting its stated goals? 



Very successful 


Somewhat 

successful 


Somewhat 

unsuccessful 


Not 

successful 


Don’t know 


25.0(7) 


64.3% (18) 


3.6% (1) 


0.0 


7.1% (2) 



Were the goals of the program properly matched with the needs of the students? Why, or 
why not? 



TEACHERS’ RESPONSES 


PERCENT 


NUMBERS 


Yes 


88.9% 


24 


No 


3.7% 


1 


Don’t Know 


7.4 


2 



Explanations. 

Teachers were able to give students more individual attention. 
Pretesting allowed instructional level learning to take place earlier. 
Students developed literacy strategies. 

Running records indicated students’ progress. 

Students gained more independence. 
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13 . Of the professional resources and support available to you, which were necessary to make this 

program a success? What other resources/support did you need? 



RESOURCES AVAILABLE 


RESOURCES NEEDED 


Parent-Training Specialists. 


More material. 


Lead teacher and other teachers w/prior program experience. 


Better communication w/central office. 


Training workshops. 


More support before the program began. 


Planning time. 




Campus-level choice of materials. 


All of the resources available were necessary. 



14 . Will you use the portfolios and/or other teaching methods advocated by this program in your regular 
classes in the future? Why, or why not? 



TEACHERS’ RESPONSES 


PERCENT 


NUMBERS 


Yes 


92.8% 


26 


No 


3.6% 


1 


Don’t know 


3.6% 


1 



Explanations: 

Portfolios help keep records. 

Sending reading materials home daily was very effective. 
The portfolios and teaching methods “work.” 
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Accelerating Literacy Program (ALP) - Summer 1995 
Parent Training Specialist Survey 



1 . What kinds of support did you offer students to facilitate their participation in the Accelerating 
Literacy Program (ALP)? 



PARENT-TRAINING SPECIALISTS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Attendance. 


37.5% 


3 


Home visits. 


25.0% 


2 


Worked one-on-one with the students. 


37.5% 


3 


Provided transportation for students. 


12.5% 


1 


Provided incentives for students. 


37.5% 


3 



2 . 



How many of the students had to be picked up from home more than twice weekly to attend ALP 
classes? 



AVERAGE 



3 



3. How much of your time was spent picking up students at home to bring them to class? (Circle 
one) 



100%-90% 


89%-75% 


74%-50% 


49%-25% 


less than 25% 


0.0 


0.0 


25.0% (2) 


25.0% (2) 


50.0% (4) 
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4. What kinds of parent involvement were encouraged (e g., parent training meeting, teacher 
conferences, classroom volunteer, etc.)? 



PARENT-TRAINING SPECIALISTS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Parent workshops. 


87.5% 


7 


Classroom visitations. 


12.5% 


1 


Survey. 


12.5% 


1 



5. How were parents contacted? What kinds of support did you offer parents to facilitate their 



participation in ALP? 



PARENT-TRAINING SPECIALISTS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Letters. 


62.5% 


5 


Pamphlets. 


25.0% 


2 


Home visits. 


50.0% 


4 


Telephone. 


75% 


6 


Teachers did home visits.. 


12.5% 


1 


Newsletter. 


12.5% 


1 


Offered parents’ transportation to/from parent meetings. 


12.5% 


1 



6. How are the methods used by ALP to encourage parental involvement more or less effective 
than other methods used in the District? 



PARENT-TRAINING SPECIALISTS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


Don’t know/same. 


37.5% 


3 


Great having daily contact with parents. 


37.5% 


3 


Methods were harder to use due to half-dav schedule. 


12.5% 


1 
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7. Of the professional resources and support available to you, which were necessary to make this 
program a success? What other resources/support did you need? 



PARENT-TRAINING SPECIALISTS’ RESPONSES 


PERCENT 


NUMBERS 
(multiple responses) 


NEEDS: 






No guidelines from District were available. 


37.5% 


3 


Needed buses. 


12.5% 


1 


NECESSARY RESOURCES: 






RIF books - for incentives. 


12.5% 


1 


Adopters provided food, clothing, etc. for students when necessary 


12.5% 


1 


Good communication with teachers. 


12.5% 


1 
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Scale of Writing Development 



Level 1 The child- 

attempts to write in scribbles or draws patterns. 



Level 2 The child- 

writes alphabet and mock letters scattered around the page, 
writes mock letters, 
pretends to write. 



Level 3 The child- 

copies words s\he sees around the room. 

writes alphabet letters and mock letters in a line across the page. 



Level 4 The child- 

repeats message. 

has message concept and tells you what the message is. 
uses letters that don't match sounds, 
writes alphabet letter strings. 



Level 5 The child- 

labels drawings. 

uses letters that have some connection to sounds, 
writes lists. 

separates words with a space or marker, 
writes a message, 
writes familiar words. 



Level 6 The child- 

invents spellings. 

presents a story that is single factual statement, 
presents a message that is understandable. 



Level 7 The child- 

writes the start of a story. 

uses both phonics and sight strategies to spell words, 
writes several short sentences. 



Level 8 The child- 

writes a short story with a beginning, middle, and an end. 

writes for several different purposes (narrative, expository, and persuasive). 

revises by adding to the story. 

begins to use punctuation. 



Level 9 The child- 

includes detail or dialogue, a sense or humor or other emotions, 
retells a familiar story or follows the pattern of a known story or poem, 
uses more conventional spelling, 
willingly revises. 

Level 10 The child- 

willingly revises and edits. 

writes creatively and imaginatively. 

writes original poetry. 

writes clearly (the message makes sense). 

uses commas, quotation marks, apostrophes. 

Level 11 The child- 

uses a variety of strategies for revision and editing. 

uses writing techniques to build suspense, create humor, etc. 

" One Schools Adventure in Portfolio Analysis” by L. Lamme and C. Hysmith The Whole Language Catalogue Supplement on Authentic Assessment p& 122 
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